Decentralized Riemannian Algorithm for Nonconvex Minimax Problems

The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has been actively applied to solve many problems, such as robust dimensionality reduction and deep neural networks with orthogonal weights (Stiefel manifold). Although algorithms for problems have developed in the Euclidean setting, it is difficult convert them into cases, constraints are even rare. On other hand, address big data challenges, decentralized (serverless) training techniques recently emerging since they can reduce communications overhead avoid bottleneck problem on server node. Nonetheless, algorithm not studied. In this paper, we study distributed nonconvex-strongly-concave Stiefel manifold propose both deterministic stochastic methods. Steifel a non-convex set. global function represented finite sum of local functions. For DRGDA prove that our method achieves gradient complexity O( epsilon(-2)) under mild conditions. DRSGDA epsilon(-4)). first exact convergence. Extensive experimental results Deep Neural Networks (DNNs) demonstrate efficiency algorithms.